Using Virtual Document for NTCIR-4 Web Information Retrieval Task

The Web is a large collection of heterogeneous pages. Web documents are not always descriptive and accurate in content. In addition, a significant difference between the problems of Web search and traditional text search is the availability of hyperlinks between pages. A page on the Web might possibly be cited by or cite other pages. When evaluating a page, the neighborhood of the page might be a part of the input. In this paper, in addition to the explicit information unit (page content), a new information unit, a virtual document, is introduced in our systems, which is mainly organized by the associated anchor-text of in-bounds links to a page and also its title data. We analyzed the utility of virtual document for Web searching. Three searching function based on virtual document are proposed in our study: • We propose a way to weight query terms through term entropy in the virtual document collection space. • Ranking algorithms are configured to index virtual documents as separate queryable data and our system considers it an important indicator of page relevance in addition to the relevance score of practical Web documents. • Our system implements a modified version of link analysis which employs literal matching between information units of the NTCIR-4 Web data. The experiment results show: Our Web searching system that uses our proposed ranking function works well. The new information units, virtual documents, play an important role in improving information retrieval results. Query term weighting using term entropy on virtual document space is effective in improving searching results. The combination of evidence from actual documents and virtual documents can improve searching results beyond either information source alone. The Query-independent score, which is calculated by our proposed link analysis model, could also obtain modest improvements through our tentative re-ranking methods.